Biostat 203B Homework 1

Due Jan 24, 2025 @ 11:59PM

Author

Jiaye Tian 306541095

Display machine information for reproducibility:

sessionInfo()
R version 4.4.1 (2024-06-14)
Platform: aarch64-apple-darwin20
Running under: macOS Sonoma 14.0

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/lib/libRblas.0.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] htmlwidgets_1.6.4 compiler_4.4.1    fastmap_1.2.0     cli_3.6.3        
 [5] tools_4.4.1       htmltools_0.5.8.1 rstudioapi_0.16.0 yaml_2.3.10      
 [9] rmarkdown_2.28    knitr_1.48        jsonlite_1.8.9    xfun_0.49        
[13] digest_0.6.37     rlang_1.1.4       evaluate_1.0.1   

Q1. Git/GitHub

No handwritten homework reports are accepted for this course. We work with Git and GitHub. Efficient and abundant use of Git, e.g., frequent and well-documented commits, is an important criterion for grading your homework.

  1. Apply for the Student Developer Pack at GitHub using your UCLA email. You’ll get GitHub Pro account for free (unlimited public and private repositories).

  2. Create a private repository biostat-203b-2025-winter and add Hua-Zhou and TA team (Tomoki-Okuno for Lec 1; parsajamshidian and BowenZhang2001 for Lec 82) as your collaborators with write permission.

  3. Top directories of the repository should be hw1, hw2, … Maintain two branches main and develop. The develop branch will be your main playground, the place where you develop solution (code) to homework problems and write up report. The main branch will be your presentation area. Submit your homework files (Quarto file qmd, html file converted by Quarto, all code and extra data sets to reproduce results) in the main branch.

  4. After each homework due date, course reader and instructor will check out your main branch for grading. Tag each of your homework submissions with tag names hw1, hw2, … Tagging time will be used as your submission time. That means if you tag your hw1 submission after deadline, penalty points will be deducted for late submission.

  5. After this course, you can make this repository public and use it to demonstrate your skill sets on job market.

Solution Q1 is completed.

Q2. Data ethics training

This exercise (and later in this course) uses the MIMIC-IV data v3.1, a freely accessible critical care database developed by the MIT Lab for Computational Physiology. Follow the instructions at https://mimic.mit.edu/docs/gettingstarted/ to (1) complete the CITI Data or Specimens Only Research course and (2) obtain the PhysioNet credential for using the MIMIC-IV data. Display the verification links to your completion report and completion certificate here. You must complete Q2 before working on the remaining questions. (Hint: The CITI training takes a few hours and the PhysioNet credentialing takes a couple days; do not leave it to the last minute.)

Solution Here is the link to my Completion Report. Here is the link to my Completion Certification.

Q3. Linux Shell Commands

  1. Make the MIMIC-IV v3.1 data available at location ~/mimic. The output of the ls -l ~/mimic command should be similar to the below (from my laptop).
# content of mimic folder
# ls -l ~/mimic/

Refer to the documentation https://physionet.org/content/mimiciv/3.1/ for details of data files. Do not put these data files into Git; they are big. Do not copy them into your directory. Do not decompress the gz data files. These create unnecessary big files and are not big-data-friendly practices. Read from the data folder ~/mimic directly in following exercises.

Use Bash commands to answer following questions.

  1. Display the contents in the folders hosp and icu using Bash command ls -l. Why are these data files distributed as .csv.gz files instead of .csv (comma separated values) files? Read the page https://mimic.mit.edu/docs/iv/ to understand what’s in each folder.

  2. Briefly describe what Bash commands zcat, zless, zmore, and zgrep do.

  3. (Looping in Bash) What’s the output of the following bash script?

for datafile in ~/mimic/hosp/{a,l,pa}*.gz
do
  ls -l $datafile
done

Display the number of lines in each data file using a similar loop. (Hint: combine linux commands zcat < and wc -l.)

  1. Display the first few lines of admissions.csv.gz. How many rows are in this data file, excluding the header line? Each hadm_id identifies a hospitalization. How many hospitalizations are in this data file? How many unique patients (identified by subject_id) are in this data file? Do they match the number of patients listed in the patients.csv.gz file? (Hint: combine Linux commands zcat <, head/tail, awk, sort, uniq, wc, and so on.)

  2. What are the possible values taken by each of the variable admission_type, admission_location, insurance, and ethnicity? Also report the count for each unique value of these variables in decreasing order. (Hint: combine Linux commands zcat, head/tail, awk, uniq -c, wc, sort, and so on; skip the header line.)

  3. The icusays.csv.gz file contains all the ICU stays during the study period. How many ICU stays, identified by stay_id, are in this data file? How many unique patients, identified by subject_id, are in this data file?

  4. To compress, or not to compress. That’s the question. Let’s focus on the big data file labevents.csv.gz. Compare compressed gz file size to the uncompressed file size. Compare the run times of zcat < ~/mimic/labevents.csv.gz | wc -l versus wc -l labevents.csv. Discuss the trade off between storage and speed for big data files. (Hint: gzip -dk < FILENAME.gz > ./FILENAME. Remember to delete the large labevents.csv file after the exercise.)

Q5. More fun with Linux

Try following commands in Bash and interpret the results: cal, cal 2025, cal 9 1752 (anything unusual?), date, hostname, arch, uname -a, uptime, who am i, who, w, id, last | head, echo {con,pre}{sent,fer}{s,ed}, time sleep 5, history | tail.

cal

Displays the calendar of the current month. Output A formatted calendar of the current month with the current date marked.

cal 2025
                            2025
      January               February               March          
Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  
          1  2  3  4                     1                     1  
 5  6  7  8  9 10 11   2  3  4  5  6  7  8   2  3  4  5  6  7  8  
12 13 14 15 16 17 18   9 10 11 12 13 14 15   9 10 11 12 13 14 15  
19 20 21 22 23 24 _2_5  16 17 18 19 20 21 22  16 17 18 19 20 21 22  
26 27 28 29 30 31     23 24 25 26 27 28     23 24 25 26 27 28 29  
                                            30 31                 

       April                  May                   June          
Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  
       1  2  3  4  5               1  2  3   1  2  3  4  5  6  7  
 6  7  8  9 10 11 12   4  5  6  7  8  9 10   8  9 10 11 12 13 14  
13 14 15 16 17 18 19  11 12 13 14 15 16 17  15 16 17 18 19 20 21  
20 21 22 23 24 25 26  18 19 20 21 22 23 24  22 23 24 25 26 27 28  
27 28 29 30           25 26 27 28 29 30 31  29 30                 
                                                                  

        July                 August              September        
Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  
       1  2  3  4  5                  1  2      1  2  3  4  5  6  
 6  7  8  9 10 11 12   3  4  5  6  7  8  9   7  8  9 10 11 12 13  
13 14 15 16 17 18 19  10 11 12 13 14 15 16  14 15 16 17 18 19 20  
20 21 22 23 24 25 26  17 18 19 20 21 22 23  21 22 23 24 25 26 27  
27 28 29 30 31        24 25 26 27 28 29 30  28 29 30              
                      31                                          

      October               November              December        
Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  
          1  2  3  4                     1      1  2  3  4  5  6  
 5  6  7  8  9 10 11   2  3  4  5  6  7  8   7  8  9 10 11 12 13  
12 13 14 15 16 17 18   9 10 11 12 13 14 15  14 15 16 17 18 19 20  
19 20 21 22 23 24 25  16 17 18 19 20 21 22  21 22 23 24 25 26 27  
26 27 28 29 30 31     23 24 25 26 27 28 29  28 29 30 31           
                      30                                          

Displays the calendar of the entire year 2025. Output A grid with all the months in 2025.

cal 9 1752
   September 1752     
Su Mo Tu We Th Fr Sa  
       1  2 14 15 16  
17 18 19 20 21 22 23  
24 25 26 27 28 29 30  
                      
                      
                      

Displays the calendar for September 1752.

date
Sat Jan 25 01:47:55 EST 2025

Displays the current date and time. Output: Current system time in the format

hostname
Justinas-MacBook-Pro.local

Shows the name of the computer or network hostname. Output: The hostname of your machine.

arch
arm64

Displays the architecture of your system (e.g., x86_64, arm64). Output: System architecture.

uname -a
Darwin Justinas-MacBook-Pro.local 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:34 PDT 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T8103 arm64

Displays detailed information about the system kernel, architecture, and operating system.

uptime
 1:47  up 56 days, 10:50, 1 user, load averages: 6.38 5.89 5.72

Shows how long the system has been running, the number of active users, and system load averages.

who am i
justinatian                   Jan 25 01:47 

Shows the logged-in user and their session details. Output: Current user information

who
justinatian      console      Jan 22 17:36 

Lists all users currently logged into the system. Output: Multiple lines showing usernames, terminals, and login times.

w
 1:47  up 56 days, 10:50, 1 user, load averages: 6.38 5.89 5.72
USER     TTY      FROM              LOGIN@  IDLE WHAT
justinatian console  -                Wed17   2days -

Displays who is logged in and what they are doing. Output: A summary including uptime, number of users, and their processes.

id
uid=501(justinatian) gid=20(staff) groups=20(staff),12(everyone),61(localaccounts),79(_appserverusr),80(admin),81(_appserveradm),98(_lpadmin),701(com.apple.sharepoint.group.1),33(_appstore),100(_lpoperator),204(_developer),250(_analyticsusers),395(com.apple.access_ftp),398(com.apple.access_screensharing),399(com.apple.access_ssh),400(com.apple.access_remote_ae)

Shows the user ID (UID) and group ID (GID) of the current user.

last | head
justinatian console                         Wed Jan 22 17:36   still logged in
justinatian console                         Sun Jan 12 11:34 - 06:41  (19:07)
justinatian console                         Sat Jan 11 21:59 - 02:55  (04:56)
justinatian ttys000                         Sat Jan 11 18:31 - 18:31  (00:00)
justinatian ttys000                         Sat Jan 11 17:26 - 17:26  (00:00)
justinatian console                         Sat Jan 11 16:01 - 20:27  (04:25)
justinatian console                         Sat Jan 11 11:51 - 15:59  (04:07)
justinatian console                         Fri Jan 10 00:55 - 03:30  (02:35)
justinatian ttys000                         Thu Jan  9 22:56 - 22:56  (00:00)
justinatian ttys000                         Thu Jan  9 22:56 - 22:56  (00:00)

Displays the login history of users. | head limits the output to the first 10 lines. Output: A list of recent logins

echo {con,pre}{sent,fer}{s,ed}
consents consented confers confered presents presented prefers prefered

Generates all possible combinations of words formed by the braces.

time sleep 5

real    0m5.008s
user    0m0.000s
sys 0m0.002s

Measures the time taken to execute the command sleep 5, which pauses for 5 seconds. Output: Real, user, and system time for the command execution

history | tail

Displays the last 10 commands in the command history. Output: List of the last 10 commands you executed in the terminal.

Solution Q5 is completed.

Q6. Book

  1. Git clone the repository https://github.com/christophergandrud/Rep-Res-Book for the book Reproducible Research with R and RStudio to your local machine. Do not put this repository within your homework repository biostat-203b-2025-winter.

  2. Open the project by clicking rep-res-3rd-edition.Rproj and compile the book by clicking Build Book in the Build panel of RStudio. (Hint: I was able to build git_book and epub_book directly. For pdf_book, I needed to add a line \usepackage{hyperref} to the file Rep-Res-Book/rep-res-3rd-edition/latex/preabmle.tex.)

The point of this exercise is (1) to obtain the book for free and (2) to see an example how a complicated project such as a book can be organized in a reproducible way. Use sudo apt install PKGNAME to install required Ubuntu packages and tlmgr install PKGNAME to install missing TexLive packages.

For grading purpose, include a screenshot of Section 4.1.5 of the book here.

Solution ::: {figure} pdf screenshot :::

html screenshot

epub The Books app on macOS can open the EPUB file, so I used it and took a screenshot to demonstrate my work.